Engineering Trustworthy Self-Adaptive Software with Dynamic Assurance Cases

Building on concepts drawn from control theory, self-adaptive software handles environmental and internal uncertainties by dynamically adjusting its architecture and parameters in response to events such as workload changes and component failures. Self-adaptive software is increasingly expected to meet strict functional and non-functional requirements in applications from areas as diverse as manufacturing, healthcare and finance. To address this need, we introduce a methodology for the systematic ENgineering of TRUstworthy Self-adaptive sofTware (ENTRUST). ENTRUST uses a combination of (1) design-time and runtime modelling and verification, and (2) industry-adopted assurance processes to develop trustworthy self-adaptive software and assurance cases arguing the suitability of the software for its intended application. To evaluate the effectiveness of our methodology, we present a tool-supported instance of ENTRUST and its use to develop proof-of-concept self-adaptive software for embedded and service-based systems from the oceanic monitoring and e-finance domains, respectively. The experimental results show that ENTRUST can be used to engineer self-adaptive software systems in different application domains and to generate dynamic assurance cases for these systems

A Case Study on Formal Verification of Self-Adaptive Behaviors in a Decentralized System

Self-adaptation is a promising approach to manage the complexity of modern software systems. A self-adaptive system is able to adapt autonomously to internal dynamics and changing conditions in the environment to achieve particular quality goals. Our particular interest is in decentralized self-adaptive systems, in which central control of adaptation is not an option. One important challenge in self-adaptive systems, in particular those with decentralized control of adaptation, is to provide guarantees about the intended runtime qualities. In this paper, we present a case study in which we use model checking to verify behavioral properties of a decentralized self-adaptive system. Concretely, we contribute with a formalized architecture model of a decentralized traffic monitoring system and prove a number of self-adaptation properties for flexibility and robustness. To model the main processes in the system we use timed automata, and for the specification of the required properties we use timed computation tree logic. We use the Uppaal tool to specify the system and verify the flexibility and robustness properties.Comment: In Proceedings FOCLASA 2012, arXiv:1208.432

Assuring system goals under uncertainty with active formal models of self-adaptation

Designing software systems with uncertainties, such as incomplete knowledge about changing system goals, is challenging. One approach to handle uncertainties is self-adaptation, where a system consists of a managed system and a managing system that realizes a feedback loop. The promise of self-adaptation is to enable a system to adapt itself realizing the system goals, regarding uncertainties. To realize this promise it is critical to provide assurances for the self-adaptive behaviors. Several approaches have been proposed that exploit formal methods to provide these assurances. However, an integrated approach that combines: (1) seamless integration of offline and online verification (to deal with inherent limitations of verification), with (2) support for runtime evolution of the system (to deal with new or changing goals) is lacking. In this paper, we outline a new approach named Active FORmal Models of Selfadaptation (ActivFORMS) that aims to deal with these challenges. In ActivFORMS, the formal models of the managing system are directly deployed and executed to realize self-adaptation, guaranteeing the verified properties. Having the formal models readily available at runtime paves the way for: (1) incremental verification during system execution, and (2) runtime evolution of the self-adaptive system. Experiences with a robotic system show promising results. Copyright © 2014 ACM.status: publishe

ActivFORMS: active formal models for self-adaptation

Self-adaptation enables a software system to deal autonomously with uncertainties, such as dynamic operating conditions that are difficult to predict or changing goals. A common approach to realize self-adaptation is with a MAPE-K feedback loop that consists of four adaptation components: Monitor, Analyze, Plan, and Execute. These components share Knowledge models of the managed system, its goals and environment. To provide guarantees of the adaptation goals, state of the art approaches propose using formal models of the knowledge. However, less attention is given to the formalization of the adaptation components themselves, which is important to provide guarantees of correctness of the adaptation behavior (e.g., does the execute component execute the plan correctly?). We propose Active FORmal Models for Self-adaptation (ActivFORMS) that uses an integrated formal model of the adaptation components and knowledge models. The formal model is directly executed by a virtual machine to realize adaptation, hence active model. The contributions of ActivFORMS are: (1) the approach assures that the adaptation goals that are verified offline are guaranteed at runtime, and (2) it supports dynamic adaptation of the active model to support changing goals. We show how we have applied ActivFORMS for a small-scale robotic system.status: publishe

ActivFORMS: A Formally-Founded Model-Based Approach to Engineer Self-Adaptive Systems

Self-adaptation enables a software system to deal with uncertainties that are difficult to anticipate before deployment, such as dynamic availability of resources and fluctuating workloads. Self-adaptation is realized by adding a feedback loop to the system that collects runtime data to resolve the uncertainties and adapts the system to realize its goals (i.e., adaptation goals). A common approach to ensure that the system complies with the adaptation goals is using formal techniques at runtime. Yet, existing approaches have three limitations that affect their practical applicability: (i) they ignore correctness of the behavior of the feedback loop, (ii) they rely on exhaustive verification at runtime to select adaptation options to realize the adaptation goals, which is time and resource demanding, and (iii) they provide limited or no support for changing adaptation goals at runtime. To tackle these shortcomings, we contribute ActivFORMS (Active FORmal Models for Self-adaptation), a reusable end-to-end approach for engineering self-adaptive systems that spans the design, deployment, runtime adaptation, and evolution of a feedback loop. We also contribute ActivFORMSi, a tool-supported instance of ActivFORMS. The approach relies on formally verified models that are directly deployed and executed using a model execution engine. At runtime the feedback loop selects adaptation options that realize the adaptation goals in an efficient manner using statistical model checking. The approach offers basic support for changing adaptation goals and evolving verified models of the feedback. We validate the approach for an IoT application for building security monitoring deployed in Leuven. The results demonstrate that the approach supports correct behavior of the feedback loop, efficiently achieves the adaptation goals, and supports changing adaptation goals at runtime, for a practical system.Comment: 40 page

Model checking of self-adaptive behaviours in a multi-agent system for traffic monitoring

status: publishe

Towards runtime statistical model checking for self-adaptive systems

With the increasing demand for self-adaptation in applications with critical goals, providing guarantees for these goals at runtime has become an important subject of research. One of the prominent proposed approaches is automated verification at runtime that allows verifying goals on the fly, typically by exhaustive traversal of the state graph of the system model. However, this approach suffers from the well-known state space explosion problem. We put forward runtime statistical model checking (RSMC) as an efficient alternative to provide guarantees for self-adaptive systems. Using statistical methods, RSMC enables the system to verify properties at runtime with a required accuracy and level of confidence. An important benefit of RSMC is that it allows to tradeoff between the accuracy and confidence of the guarantees it provides with the computation time and system resources it requires. We provide a model for RSMC in self-adaptive systems based on MAPE-based feedback loops and illustrate the benefits of the approach using the Tele Assistance System exemplar.nrpages: 31status: publishe

SimCA vs ActivFORMS: comparing control- and architecture-based adaptation on the TAS exemplar

© 2015 ACM. Today customers require software systems to provide particular levels of qualities, while operating under dynamically changing conditions. These requirements can be met with different self-adaptation approaches. Recently, we developed two approaches that are different in nature - control theory-based SimCA and architecture-based ActivFORMS - to endow software systems with self-adaptation, providing guarantees on desired behavior. However, it is unclear which of the two approaches should be used in different adaptation scenarios and how effective they are in comparison to each other. In this paper, we apply SimCA and ActivFORMS to the Tele Assistance System (TAS) exemplar and compare obtained results, demonstrating the difference in achieved qualities and formal guarantees.status: publishe

Do external feedback loops improve the design of self-adaptive systems? A controlled experiment

Providing high-quality software in the face of uncertainties, such as dealing with new user needs, changing availability of resources, and faults that are difficult to predict, raises fundamental challenges to software engineers. These challenges have motivated the need for self-adaptive systems. One of the primary claimed benefits of self-adaptation is that a design with external feedback loops provide a more effective engineering solution for self-adaptation compared to a design with internal mechanisms. While many efforts indicate the validity of this claim, to the best of our knowledge, no controlled experiments have been performed that provide scientifically founded evidence for it. Such experiments are crucial for researchers and engineers to underpin their claims and improve research. In this paper, we report the results of a controlled experiment performed with 24 final-year students of a Master in Software Engineering program in which designs based on external feedback loops are compared with designs based on internal mechanisms. The results show that applying external feedback loops can reduce control flow complexity and fault density, and improve productivity. We found no evidence for a reduction of activity complexity. © 2013 IEEE.status: publishe

Claims and supporting evidence for self-adaptive systems: A literature study

Despite the vast body of work on self-adaption, no systematic study has been performed on the claims associated with self-adaptation and the evidence that exists for these claims. As such an insight is crucial for researchers and engineers, we performed a literature study of the research results from SEAMS since 2006 and the associated Dagstuhl seminar in 2008. The study shows that the primary claims of self-adaptation are improved flexibility, reliability, and performance of the system. On the other hand, the tradeoffs implied by self-adaptation have not received much attention. Evidence is obtained from basic examples, or simply lacking. Few systematic empirical studies have been performed, and no industrial evidence is reported. From the study, we offer the following recommendations to move the field forward: to improve evaluation, researchers should make their assessment methods, tools and data publicly available; to deal with poor discussion of limitations, conferences/workshops should require an explicit section on limitations in engineering papers; to improve poor treatment of tradeoffs, this aspect should be an explicit subject of reviews; and finally, to enhance industrial validation, the best academy-industry efforts could be formally recognized by the community. © 2012 IEEE.status: publishe